System and method for performing mobility management using haptic guidance

ABSTRACT

A method for managing movement includes determining a current location of a subject in a monitoring area, comparing the current location to a risk area location in the monitoring area, determining a likelihood of injury based on a result of the comparison, and generating control information based on the likelihood of injury to the subject. The control information may control activation of a haptic effect in a device worn or carried by the subject. The haptic effect may correspond to at least one stimulus that notifies the subject of the potential risk area.

CROSS-REFERENCE TO PRIOR APPLICATIONS

This application claims the benefit of U.S. Provisional Application No.62/949546, filed on 18 Dec. 2019. This application is herebyincorporated by reference herein.

TECHNICAL FIELD

One or more embodiments described herein generally to processinginformation, and more specifically, but not exclusively, to performingmobility management and risk avoidance.

BACKGROUND

The aging population has grown by 28% from 2004 to 2014 (46.2 million)and is projected to grow to 82.3 million by the year of 2040. Accordingto the American Association of Retired Persons (AARP), 90% of seniors(65+ years) prefer to age in place, e.g., stay at home instead of movingto an assisted living or nursing facility. The burgeoning olderpopulation necessitates development of technologies that enableindependent and healthy aging-in-place, and especially technologies thatcan improve mobility and reduce mobility-related risks.

Falls are one of the principal risks to elderly people living at home,and in fact are the leading injury-related cause of death in individualsage 65 and over in the United States, according to CDC statistics. Inmost cases, the potential for a fall results from mobility impairmentdue to age-related physical decline and environmental hazards. This maylead to a wide range of negative psychological and physical outcomes.Hearing and vision impairments may only exacerbate the mobility risks.All of these factors increase lead to activity avoidance, which, inturn, adversely affects quality of life.

Current technologies focus on detection of adverse mobility events suchas personal emergency response systems (PERS, e.g., Philips lifeline®).These systems track mobility using inertial measurement unit, examplesof which include accelerometers, gyroscopes, and a magnetometers.However, these systems are unable to address many concerns relating tofalls or other types of risks. Moreover, these systems do not determineand warn of potential risks, nor do they provide help in guiding aperson to escape such risks in order to avoid an impending injury.

SUMMARY

A brief summary of various example embodiments is presented below. Somesimplifications and omissions may be made in the following summary,which is intended to highlight and introduce some aspects of the variousexample embodiments, but not to limit the scope of the invention.Detailed descriptions of example embodiments adequate to allow those ofordinary skill in the art to make and use the inventive concepts willfollow in later sections.

In accordance with one or more embodiments, a method for managingmovement, comprising determining a current location of a subject in amonitoring area; comparing the current location to a risk area locationin the monitoring area; determining a likelihood of injury based on aresult of the comparison; and generating control information based onthe likelihood of injury to the subject, wherein the control informationis to control activation of a haptic effect in a device worn or carriedby the subject and wherein the haptic effect corresponds to at least onestimulus that notifies the subject of the potential risk area.

Determining the current location may include receiving one or morelocation signals from a corresponding number of location sensors in themonitoring area; accessing layout information corresponding to themonitoring area; comparing location information corresponding to the oneor more location signals to the layout information, and determining thecurrent location of the subject based on a result of the comparison ofthe location information to the layout information. The method mayinclude generating at least one of spatial information or movementinformation, wherein the spatial information indicates a distancebetween the current location of the subject and the risk area locationand wherein the movement information indicates at least one of adirection of movement of the subject, a time of movement of the subject,or a speed of movement of the subject, and wherein the likelihood ofinjury is determined based on at least one of the spatial information orthe movement information.

The control information may indicate at least one of a type of a hapticactuator in the device, a pattern of activation of the haptic actuator,a duration of activation of the haptic actuator, or a time of activationor deactivation of the haptic actuator. The method may include accessinghaptic information correlating different types of haptic effects todifferent likelihoods of injury, wherein the control information isgenerated based on the haptic information. The risk area location mayinclude a set of stairs. The control information may controls activationof the haptic effect to alert the subject of a remaining number of stepswhen the subject is on the set of stairs.

In accordance with one or more embodiments, an apparatus for managingmovement includes a memory configured to store instructions and aprocessor configured to execute the instructions to: determine a currentlocation of a subject in a monitoring area; compare the current locationto a risk area location in the monitoring area; determine a likelihoodof injury based on a result of the comparison; and generate controlinformation based on the likelihood of injury to the subject, whereinthe control information is to control activation of a haptic effect in adevice worn or carried by the subject and wherein the haptic effectcorresponds to at least one stimulus that notifies the subject of thepotential risk area.

The processor may determine the current location by receiving one ormore location signals from a corresponding number of location sensors inthe monitoring area; accessing layout information corresponding to themonitoring area; comparing location information corresponding to the oneor more location signals to the layout information, and determining thecurrent location of the subject based on a result of the comparison ofthe location information to the layout information.

The processor may generate at least one of spatial information ormovement information, wherein the spatial information indicates adistance between the current location of the subject and the risk arealocation and wherein the movement information indicates at least one ofa direction of movement of the subject, a time of movement of thesubject, or a speed of movement of the subject, and wherein thelikelihood of injury is determined based on at least one of the spatialinformation or the movement information. The control information mayindicate at least one of: a type of a haptic actuator in the device, apattern of activation of the haptic actuator, a duration of activationof the haptic actuator, or a time of activation or deactivation of thehaptic actuator.

The processor may access haptic information correlating different typesof haptic effects to different likelihoods of injury, wherein thecontrol information is generated based on the haptic information. Therisk area location may include a set of stairs. The control informationmay control activation of the haptic effect to alert the subject of aremaining number of steps when the subject is on the set of stairs. Theapparatus may include one or more actuators to generate the at least onestimulus. The apparatus may be included in a device carried or worn bythe subject.

In accordance with one or more embodiments, a computer-readable mediumstores instructions for causing a processor to: determine a currentlocation of a subject in a monitoring area; compare the current locationto a risk area location in the monitoring area; determine a likelihoodof injury based on a result of the comparison; and generate controlinformation based on the likelihood of injury to the subject, whereinthe control information is to control activation of a haptic effect in adevice worn or carried by the subject and wherein the haptic effectcorresponds to at least one stimulus that notifies the subject of thepotential risk area.

The computer-readable medium may store instructions to cause theprocessor is to determine the current location by: receiving one or morelocation signals from a corresponding number of location sensors in themonitoring area; accessing layout information corresponding to themonitoring area; comparing location information corresponding to the oneor more location signals to the layout information, and determining thecurrent location of the subject based on a result of the comparison ofthe location information to the layout information.

The computer-readable medium may store instructions to cause theprocessor to: generate at least one of spatial information or movementinformation, wherein the spatial information indicates a distancebetween the current location of the subject and the risk area locationand wherein the movement information indicates at least one of adirection of movement of the subject, a time of movement of the subject,or a speed of movement of the subject, and wherein the likelihood ofinjury is determined based on at least one of the spatial information orthe movement information. The control information may indicate at leastone of: a type of a haptic actuator in the device, a pattern ofactivation of the haptic actuator, a duration of activation of thehaptic actuator, or a time of activation or deactivation of the hapticactuator.

BRIEF DESCRIPTION OF THE DRAWINGS

The accompanying figures, where like reference numerals refer toidentical or functionally similar elements throughout the separateviews, together with the detailed description below, are incorporated inand form part of the specification, and serve to further illustrateexample embodiments of concepts found in the claims and explain variousprinciples and advantages of those embodiments.

These and other more detailed and specific features are more fullydisclosed in the following specification, reference being had to theaccompanying drawings, in which:

FIG. 1 illustrates an embodiment a system for performing mobilitymanagement of a subject using haptic guidance;

FIG. 2 illustrates an example of a monitoring area for the subject;

FIG. 3 illustrates an example embodiment for providing haptic guidanceon stairs; and

FIG. 4 illustrates an embodiment of a method for performing mobilitymanagement of a subject using haptic guidance.

DETAILED DESCRIPTION

It should be understood that the figures are merely schematic and arenot drawn to scale. It should also be understood that the same referencenumerals are used throughout the figures to indicate the same or similarparts.

The descriptions and drawings illustrate the principles of variousexample embodiments. It will thus be appreciated that those skilled inthe art will be able to devise various arrangements that, although notexplicitly described or shown herein, embody the principles of theinvention and are included within its scope. Furthermore, all examplesrecited herein are principally intended expressly to be for pedagogicalpurposes to aid the reader in understanding the principles of theinvention and the concepts contributed by the inventor(s) to furtheringthe art and are to be construed as being without limitation to suchspecifically recited examples and conditions. Additionally, the term,“or,” as used herein, refers to a non-exclusive or (i.e., and/or),unless otherwise indicated (e.g., “or else” or “or in the alternative”).Also, the various example embodiments described herein are notnecessarily mutually exclusive, as some example embodiments can becombined with one or more other example embodiments to form new exampleembodiments. Descriptors such as “first,” “second,” “third,” etc., arenot meant to limit the order of elements discussed, are used todistinguish one element from the next, and are generallyinterchangeable. Values such as maximum or minimum may be predeterminedand set to different values based on the application.

Example embodiments include a system and method for performing mobilitymanagement of a subject using haptic guidance. In one implementation,the system and method may determine the existence of one or more riskareas in a monitoring area of a subject and then control activation ofone or more haptic responses. The haptic responses may inform or warnthe subject of the existence of the risk area(s) and/or guide thesubject away from or through the risk area(s). Such a system may be usedby anyone for a variety of applications, but may be especiallybeneficial for the elderly who have limited mobility, persons withmental or physical disabilities, people who are blind or have some formof vision impairment, or people who are deaf or have another form ofhearing impairment.

The haptic responses may be produced by one or more actuators in adevice worn or carried by the subject. When activated, the actuator(s)generate a perceptible or tactile stimulus that warns the subject of apotential risk area, and in some embodiments also guide the subject awayfrom the risk area in order to avoid injury. These features may beaccompanied by voice-command input and/or output operations thatincrease the convenience and effectiveness of mobility management. Inone implementation, haptic responses may be generated in order to guidea person to a desired location along a route that avoids risk areas.

FIG. 1 illustrates an embodiment of a system for performing mobilitymanagement of a subject using haptic guidance. The subject may be anytype of person, but the embodiments described herein are especiallybeneficial for persons who are elderly, mentally or physicalhandicapped, hearing impaired, and vision impaired, as well as otherswho may have issues with mobility and therefore are prime candidates forfall or other types of mobility, physical, personal, and environmentalinjury risks. In some cases, the embodiments may assist persons ofyounger age who are able to understand the warnings generated forimpending risks and change behavior in accordance with the hapticguidance.

Referring to FIG. 1, the system includes a processor 10, a memory 20,and a data storage 30. The processor performs management and controloperations of the system, along with retrieval, storage, and processingof information stored in the data storage area. The memory 20 storesinstructions for controlling the processor to perform operationsdescribed herein. The instructions may be, for example, embodied as anapplication on a device, control program of a system processor, oranother form of code or software for managing the mobility of a subjectto be monitored, depending on the intended implementation of the system.The data storage area 30 may be located in the same device as theprocessor 10 or may be coupled to the processor 10 through acommunication link or network. The data storage area may be embodied ina variety of forms, including but not limited to a database or memorydevice.

The processor 10 may include a localization module 12 and a signalanalysis module 14, in this embodiment. The localization module maydetermine a current location of a subject to be monitored relative tothe location(s) of one or more features that are in the vicinity of thesubject. In one example embodiment, the localization module 12 mayreceive signals from one or more beacons (or other location signaltransmitting devices) set at predetermined locations throughout thevicinity where risk areas may be present. Examples of these risksinclude steps or stairs, protruding objects, floor hazards, clutter,furniture, house fixtures, and/or other objects that may present amobility risk in the monitoring area. The monitoring area may includeindoor and/or outdoor locations. Examples of indoor locations include ahouse, apartment, work environment, office, classroom, or other placewhere the subject is active, a fall may occur, or a mobility risk ispresent. Examples of outdoor locations include a yard, path, street,ditches, curbs, water, or other places where a fall may occur or whichotherwise may present a risk of injury.

In order to determine the location of the subject, the localizationmodule 12 may receive signals from one or more sources. The sources mayinclude one or more sensors that generate signals indicative of thelocation or movement of the subject. Examples of the sensors includeaccelerometers (e.g., carried on the patient) and pressure sensors, forexample, located on the stairs that generate signals to track the stepsof the subject on stairs. Other sources that generate location ormovement signals (collectively referred to as location signals) includebarometers and indoor localization technologies. The indoor localizationtechnologies include camera systems, motion detectors, RFID or tagtechnologies, various types of beacon systems (e.g., RF, Wi-Fi,Bluetooth, server-based, or other types of beacons), and other types oftracking or navigation systems. Examples of the beacon systems includeApply iBeacon and Google Beacon Platform (Eddystone google)).

Outdoor localization technologies include GPS, location informationgenerated from mobile communications systems, RF-based systems, camerasystems, and other types of outdoor tracking and navigation systems. Inone embodiment, a combination of the aforementioned indoor and outdoorlocation technologies. For example, barometers along with accelerometersmay be used to detect movement of a subject ascending and descending astair case, stair-by-stair. The location signals may be pre-processed bythe localization module 14 to remove noise and other spurious signalsfor improved location determination.

FIG. 2 illustrates an example of a monitoring area that includes riskareas. In this example, the monitoring area includes a layout 210indicated by information stored in the data storage area 30. The layoutinformation may serve as a frame of reference for, one, processing thelocation signals to determine the current location of the subject in themonitoring area and, two, indicating the location(s) of one or more riskarea(s) in the monitoring area. In one embodiment, other areas ofinterest (e.g., ones different from risk areas, including but notlimited to certain rooms in a house, a telephone, or another feature orobject). In addition to a floor plan, the layout information mayindicate the size, perimeter, contents, and/or other features ofinterest in the living environment of the subject.

The risk area(s) indicated in the layout information may correspond topositions of location sensors 220 ₁ to 220 ₅ arranged throughout themonitoring area. In this example, signals generated from the locationsensors are sent to a wearable device 230 on the subject 240 to bemonitored. The wearable device is depicted as a watch-based device thatreceives the location signals for generating at least one hapticresponse, in a manner to be discussed in greater detail below. In oneembodiment, the watch may bidirectionally communicate with the locationsensors for purposes of controlling the activation, operational mode, orparameters of those sensors. In FIG. 2, the layout 210 of the monitoringarea of the subject includes risk areas identified by five locationsensors 220 ₁ to 220 ₅, two of which mark respective ends of a set ofstairs 250, which may present a fall risk for the subject.

The layout information stored in the data storage area 30 may be used asa basis for controlling the activation or deactivation of one or more ofthe location sensors under predetermined circumstances, e.g., atdifferent times of the day, when the subject is detected to be at home(e.g., as determined, for example, by motion sensors), or when otherconditions have been satisfied. Thus, in these cases, placement of thelocation sensors may depend on not only the presence of certainhazardous areas or objects (e.g., stairs) in the monitoring area, butalso certain temporal patterns or physical characteristics of thesubject. These temporal patterns or physical characteristics may betaken into consideration (e.g., programmed into the control software ofthe system) for purposes of activating or deactivating the locationsensors at different times. One example relates to falls, e.g., fallsmight be more common in the morning due to the prevalence of hypotensionin the morning time. This may be taken into account when designing ordetermining the placement of the location sensors, and when determininga schedule for selectively activating certain ones of the locationsensors.

As an additional example, consider the case of a subject who has achronic condition. Storing information corresponding to the chroniccondition (e.g., in a user profile accessible by the processor 10) mayserve as a basis for creating a more user-aware early warning system.For example, subjects with Alzheimer's disease may be more prone tofalls in the evening due to sundowning syndrome. In this case, processor10 may automatically activate the location sensors in the evening. Inone implementation, the entire system may be automatically activated inthe evening and deactivated at other times, given the condition(s) ofthe subject to be monitored and/or other circumstances programmed intothe control instructions of the system.

In another embodiment, a subject profile may contain information on thetype and schedule for medications they are taking. The data storage (30)stores a mapping between medications and different side effectsincluding those effecting subject's mobility and balance. Using thisknowledge, the system may then activate and start monitoring the userand provide haptic mobility guidance following intake of medications,depending on the side effects and their durations. In anotherembodiment, subject's behavioral patterns e.g., the toileting pattern atnight-time, may be stored in subject's profile and used to determinesubject's intention and guide the subject via haptic feedback to thebathroom in the dark once awakening followed by mobility are detectedduring night-time. The processor 10 of the system may be incorporatedinto the wearable device 230 or may be integrated into a monitoringcontroller in or remotely located from the monitoring area. For example,such a monitoring controller may be located in a base station 280 (e.g.,computer, smartphone, etc.) which receives the signals directly from thelocation sensors or indirectly through the wearable device. In oneembodiment, the base station may send the signals from the locationsensors through a network 270 to a monitoring server 290 which includesthe processor 10 for performing operations of the system. When theprocessor performs selective control, the processor 10 may selectivelycontrol activation and deactivation of the location sensorsautomatically according to a programmed schedule or based on sensing thepresence of the subject in the monitoring area and/or at least partiallybased on a control signal generated by the subject, a caregiver, oranother party.

In one embodiment, the data storage area 30 may store information (e.g.,in a user profile) indicative of prior mobility incidents and/orhistorical environmental hazards relating to the subject and/or thelayout. This data may then be used by the processor 10 to develop amodel that can determine whether the subject is approaching a particularhigh-risk area in the layout (e.g., hazard area) for the subject and/orpredict a deterioration in mobility or other impairments of the subject.The deterioration may be the result of seasonal or other types oftransient declines in the condition of the subject or declines resultingfrom a change or permanent worsening of the condition. A deteriorationin the condition of the subject may, in turn, prompt a change in theactivation/deactivation schedule of the location sensors and/or warrantplacement of additional sensors at locations that were once consideredsafe but now hazardous because of a worsening of the condition of thesubject.

Once the location signals have been received from the location sensors,the localization module 12 may process those signals to determine thelocation of the subject in the monitoring area. The type of processingperformed may depend on the type(s) of location sensors used. Forexample, when the location signals are received from a set of beacons,the localization module 12 may implement a triangulation algorithm todetermine the current location of the subject (which location may changeas the subject moves) and then compare that current location to thelayout information. This comparison may then provide an indication ofthe location of the subject within the layout indicated by the layoutinformation, as illustrated, for example, in FIG. 2. In anotherembodiment, pressure sensors embedded in the stairs may be used todetermine the location of the subject in the staircase. In anotherembodiment, accelerometer and barometer signals may be used to determinethe direction, speed, and altitude of the subject, hence determiningwhether the subject is climbing or descending the stairs. Thisinformation combined with the layout of environment may be used todetermine the exact location of the subject on the staircase.Furthermore, a subject-specific mapping between sensor signals andsubject's location can be learned for every new subject. In thisembodiment, the system is configured for every new subject by collectingtest location data from deployed location sensors and mapping them tothe exact subject location conditioned on the layout of the environment.The resulting mapping may then be used to determine the location ofsubject using data streams from location sensors. This mapping may be inthe form of a regression model. For example a model may be learned todetermine the location of the subject on a staircase y(t)=f (x₁(t),x₂(t), . . . , x_(n)(t)), where y(t) is the location of subject at timet, and x_(n)(t) is the value of nth datastream (e.g., barometer'sreading) at time t. A Kalman filter model, or variants of a Kalmanfilter model, may be used to implement such a location model.

Once the current location of the subject is determined within the layoutof the monitoring area, that location may then be compared to thelocations of a respective number of predetermined risk areas in thelayout. This comparison may involve, for example, determining theproximity of the subject relative to the risk areas in terms ofdistance, time, direction of movement, speed of movement, and/or someother measure of the location of the current location of the subjectrelative to the one or more risk areas. Once this information has beendetermined, the signal analysis module 14 may determine the appropriatehaptic response for guiding or warning the subject on a real-time basis.In addition to determining location, the localization module 12 mayprocess the location signals and the layout information to track themovement of the patient in real-time.

When the localization module 12 determines that the current location ofthe subject is on a set of stairs, the movement of the subject up ordown the stairs may be tracked by comparing changes in the locationsignals with the layout information. The location signals may include,for example, beacon or motion detector signals, signals from thepressure sensors on respective ones of the steps, or a combination ofboth kinds of location signals. In this case, the layout information mayindicate that the set of stairs has ten steps. The location signals maybe compared to the layout information to determine precisely which stepthe subject is currently on, whether the subject is going up or down thesteps, how many steps the subject has already passed, and the number ofremaining stairs (NRS) the subject has to go before he reaches the endof the stairs.

The signal analysis module 14 may generate signals for providing hapticguidance to the subject based on the information output from thelocalization unit 12 indicating the current location of the subjectrelative to the one or more risk areas. The signal analysis module mayperform this operation, for example, by classifying the probabilitiesfor injury associated with the risk areas based on the current locationof the subject relative to the locations of those risk areas. In oneimplementation, the probabilities may be determined, for example, notonly based on the current location of the subject, but also the distancebetween the current location and one or more risk areas, direction ofmovement, rate of movement, types of risk areas, disabilities ordiseases of the subject, and/or other personal information stored in auser profile for the subject. A logistic regression model (feature-basedmodel) may be learned to map between subject information (e.g., currentlocation, rate and direction of movement, mental and physicalconditions) and probability of risks. Alternatively, a model thatdirectly captures temporal progression of data streams from sensors suchas a gated recurrent neural network (temporal model) may be used. Ahybrid model combining a temporal model and a feature-based model mayalso be used that receives sequences of data streams from sensors alongwith information on subject profile (e.g., current location, rate anddirection of movement, mental and physical conditions, fall history,medical conditions, time of the day, seasonality information) andestimates corresponding risk probabilities.

The signal analysis module (14) uses the estimated risk probabilities totrigger a signal for providing haptic guidance from among a set ofpredefined haptic feedbacks. This may be implemented by methodsincluding expert-authored rules (e.g. risk probabilities above/below athreshold). In one embodiment, the likelihood or severity of injury forone or more of the risk areas may be determined without calculatingformal probabilities, but rather determining into which one of aplurality of predetermined ranges the distance and/or direction ofmovement of the subject is categorized. Examples of these ranges areprovided below. In one embodiment, the signal analysis module 14 maygenerate signals for triggering at least one haptic response as thesubject is approaching a hazard area but before he actually reaches thatarea. In this case, the haptic response(s) triggered by the signalsserve as an early warning to the subject to proceed with caution or notto continue in the same direction. In the case of stairs, the signalanalysis module 14 may generate signals for triggering a haptic responsethat alerts the subject (1) that he is entering the stairs, (2) whichstep he is currently one, (3) how many remaining steps are left, and (4)that he is on the last step or has left the stairs.

Once the severity or likelihood of injury is determined, the signalanalysis module 14 determines the haptic response that corresponds tothat severity or likelihood. This determination may be made, forexample, based on information stored in data storage area 30, which mapsdifferent levels or types of severities and/or likelihoods of injury todifferent corresponding haptic responses. Such a correspondence may bestored, for example, in the form of a table or other arrangement.Examples of tables are provided and discussed in greater detail below.

In another embodiment, the signal analysis module (14) may directly mapbetween subject state (sensor data streams, subject profile) and a setof pre-defined haptic feedbacks (e.g., two short vibratory feedbackindicating of two steps left to the floor). The mapping may be learntusing a logistic regression or similar statistical models, structuredmachine learning models (e.g. gradient boosted regression trees), orsequence-based machine learning models that incorporate temporalinformation (e.g. hidden Markov models or recurrent neural networkmodels).

Once the haptic response is determined, the signal analysis module 14generates information for controlling the generation of the hapticresponse(s) on the device carried or worn by the subject. The deviceworn or carried by the subject may take various forms. For example, thedevice may be a wearable device including, but not limited to, awrist-worn device such as the watch previously discussed or a fitnesstracker, a chest-worn device such as the Philips Lifeline pendant, oranother type of device designed to be worn on the body. In anotherembodiment, the device may be a smartphone, tablet, a TV remote control,a gaming controller, or other device (implemented with an associatedapplication or as customized device) that receives signals from theprocessor.

In one embodiment, the signal analysis module 14 may determine hapticresponses to guide the subject to, not away from an area of interest.Examples of these areas include the bathroom, kitchen, front door,thermostat location, laundry area, or another location of interest. Inthis case, the signal analysis module may generate signals for providinghaptic guidance to guide the subject toward the feature of interest, forexample, in response to a request signal generated on a control device.Examples of the control device may be a smartphone application,voice-command controller (e.g., Alexa, Google Home, etc.), a smart homefeature, computer program, tablet application, or another type of device(e.g., driven by software, hardware, or both) that is configured togenerate signals requesting guidance to a feature of interest in thelayout.

The device carried or worn by the subject may include a haptic feedbackmodule 70 and one or more actuators 90, as illustrated in FIG. 1. Inorder to generate the haptic response, the haptic feedback module 70processes the control information generated by the signal analysismodule 14. This processing may include extracting information from thesignals to determine which one or combination of haptic responses are tobe generated. The extracted information may identify, for example, thetype of actuator(s) that is/are to be activated, the duration ofactivation of the actuator(s), the pattern(s) of activation, theintensity of activation, and/or other information. The pattern and/orintensity with which the one or more actuators are to be activated mayconvey information to the subject of, for example, how close inproximity he is to a risk area (or other area of interest) within themonitoring area. For example, in one implementation, the rate, period,and/or frequency of the haptic stimulus generated by the actuator maychange (e.g., increase or decrease) as the subject moves closer andcloser to the hazard area, and/or the intensity of the haptic stimulusmay change (e.g., increase or decrease). Conversely, the rate and/orintensity may change in an opposite manner as the subject moves fartheraway from the hazard area.

In one embodiment, different actuators may be triggered under differentcircumstances to convey different information to the subject. Forexample, a first actuator may be activated when the subject is enteringa set of stairs and a second actuator may be activated when the subjectis traversing up or down the stairs. In this case, the actuator maygenerate a haptic stimulus perceptible to the subject each time thesubject takes a step. In addition to the haptic response (or stimulus),the signal analysis module 14 may send control signals to the deviceworn or carried by the subject to provide an audible tone or verbalmessage of warning or other information to the subject, that may or maynot accompany the haptic response from the one or more actuators.

As previously indicated, in addition to the layout information, the datastorage area 30 may store mapping information (e.g., in table form)indicating one or more haptic responses or stimuli that may be generatedbased on the location information generated by the signal analysismodule 14. The mapping information may map location and/or locomotiveobservations that may trigger at least one actuator to generate to oneor more corresponding types of haptic feedback. In order to generate thefeedback, the haptic feedback module 70 may include or access a library(e.g., stored in the data storage area 30) that stores informationproviding a one-to-one mapping between location/locomotive observations(indicated by signals from the signal analysis module) and predeterminedhaptic feedback responses.

In one embodiment, the haptic feedback module 70 may be configured tooperate on a certain time of the day (only at night time) or anotherschedule. In these or other embodiments, the haptic feedback module mayoperate on demand, for example, in response to signals received by thesignal analysis module 14. In this case, the signals from the signalanalysis module may serve as both wake-up signals and signals containinglocation and/or motion information that is interpreted by the hapticfeedback module to generate one or more corresponding forms of hapticresponse.

Table 1 sets forth one example of the mapping that may be performed inaccordance with one embodiment. In this mapping, it is assumed that thedevice containing the haptic feedback module worn or carried by thesubject has two or more actuators, each producing a different type ofhaptic response.

TABLE 1 Subject Location Actuator 1 Actuator 2 Actuator 3 Steps ON OFFOFF Bathtub OFF ON OFF Shower Stall OFF OFF ON Non-Carpet Floor ON ONOFF Threshold Between Rooms ON OFF ON Curb Next to Mail Box OFF ON ONEdge of Deck ON ON ON

Table 2 shows additional information stored in the storage area that maybe used as a basis for mapping haptic responses. In this example, thehaptic feedback module controls a device having at least one actuator.In this case, the actuator 90 is a vibrator included in the device wornor carried by the subject being monitored. The vibrator is controlled bythe haptic feedback module to generate different patterns of vibrationbased on the signals received from the signal analysis module 14. Eachof the different patterns conveys a different type of information. Inthis example, the patterns vary based on proximity of the subject as heapproaches a hazard area, e.g., a set of stairs.

TABLE 2 Location from Stairs Vibrator Pattern Distance Range: 5-10 feetVibrating Rate: Slow Pulses Distance Range: 2-5 feet Vibrating Rate:Medium Pulses Distance Range: 0-2 feet Vibrating Rate: Fast PulsesDistance Range: 0 feet Vibrating Rate: Constant

In one embodiment, additional information may be stored in the storagearea 30 that may be used as a basis for mapping haptic responses. Inthis example, the haptic feedback module controls the performance ofhaptic responses in a device having at least one actuator. An example ofthis application is illustrated in FIG. 3, where location sensors in theform of pressure sensors 301 to 305 are positioned on respective stepsof a set of stairs 310. The haptic feedback module 70 and an actuator 90in the form of a vibrator are included in a pendant device 320 worn bythe subject 305 being monitored. The vibrator is controlled by thehaptic feedback module to generate a number of pulses of vibrations. Thenumber of vibration pulses may equal the number of remaining steps (NRS)the subject has to go before she reaches the end of the stairs.Additionally, or alternatively, the device may include a speaker whichannounces the NRS information. In one embodiment, the pressure signalsmay be coupled with signals from beacons 330 and 340 at respective endsof the staircase for providing an additional indication of the positionof the subject as she ascends or descends the stairs, and thus forproviding one or more forms of haptic warnings or guidance.

Examples of the types of actuators that may be controlled by the hapticfeedback module include, but are not limited to, ones that generatevarious types of pressures, forces and sound, in addition to thevibrations previously discussed. Examples of vibrators that are able toprovide a haptic stimulus include eccentric rotating mass, linearresonant, or piezoelectric actuators. Smartphone, smart watches areequipped with these vibratory actuators through which the hapticresponses discussed herein may be provided to and perceived by thesubject.

In one embodiment, the system for performing mobility management of asubject using haptic guidance may help navigate the subject in the dark(e.g., from his bedroom to the bathroom, or from his bedroom to thekitchen in the middle of the night). In on-demand mode, the user mayactivate the system through a software application, e.g., by activatingan application on his smartphone containing the processor (including thelocalization module and signal analysis module), the haptic feedbackmodule, and one or more actuators for generating haptic response(s). Inanother embodiment, a pendant or other device containing the hapticfeedback module (that is different from a smartphone) system may beactivated for generating warnings or other applications of the hapticstimuli. For example, in a wrist-worn embodiment, a specific handgesture (e.g., shaking the hand three times) may be linked to anactivation command that activates the system including the device. Inthis case, if the haptic feedback module is not in the same device asthe processor, then the haptic feedback module or other control logic inthe device may send a signal back to activate the processor.

FIG. 4 illustrates an embodiment of a method for performing mobilitymanagement of a subject using haptic guidance. The method may beperformed by any of the system or device embodiments described herein ormay be performed by a different system and/or device. For illustrativepurposes, the method will be described as being performed by the systemembodiments described herein.

Referring to FIG. 4, at 410, the method includes receiving, at thelocalization module 12, location signals from one or more locationsensors. The location signals may be any of the types described herein,including ones generated from Bluetooth beacons arranged atpredetermined locations corresponding to risk areas and/or other areasof interest in the monitoring area. The risk areas may include areaswhich pose a potential falling threat or other hazard, especially to asubject who is aged or has a movement, hearing or vision disability, orwho has a chronic condition. The beacons (and/or other types of locationsensors) may be near or at the area of interest. In some embodiments,multiple types of location signals may be received (e.g., beacon signalsand pressure sensor signals), for example, when the area of interestincludes a set of stairs. The beacons may be in a sleep or reduced powerstate and may be motion activated to begin transmitting location signalswhen the subject comes within a predetermined distance of thecorresponding area.

At 420, the current location of the subject is determined within themonitoring area. The current location is determined by processing thesignals from the location sensors based on layout information of themonitoring area. The type of processing that is performed may depend onthe type of location sensors or signals used. For example, sensorsignals generated by beacons may be input into a triangulation algorithmand then compared to the layout information to determine the currentlocation in the monitoring area. In another case, the distance andrelative position of the subject from a beacon, motion detector, camera,or other type of location sensor may be determined and used as a basisfor determining the current location of the subject.

At 430, the current location of the subject is compared to thelocation(s) of one or more risk areas indicated in layout information.This may involve determining the distance (e.g., closeness) between thecurrent location of the subject and the risk area and/or the degree ofproximity of the current location (e.g., different distance ranges) toone or more risk areas. For example, information indicative of thelocations of the risk areas may be stored in the layout information.Thus, once the current location of the subject is known, the processor10 may calculate the distance(s) between the current location of thesubject and the risk areas.

In addition to determining the current location, the direction and/orrate of movement of the subject may also be determined, for example, bycomparing or tracking changes in the current location of the subjectover time. The direction and/or rate of movement may provide anindication, for example, of whether the subject is approaching orleaving an area of interest, which, for example, may affect the type ofhaptic response generated as previously described.

At 440, the severity or likelihood of injury is determined based on theinformation generated in operation 430. The severity or likelihood ofinjury may be determined, for example, by comparing the distance betweenthe current location of the subject and the known locations of the riskareas (or location sensors) to a table storing a correspondence betweendifferent distances and different severities or likelihood of injury.The severity or likelihood of injury may also be determined based on thedirection or speed of movement of the subject, information of which maybe stored in the table in association with the severity or likelihood ofinjury. An example of such a table is shown below. Alternatively, theseverity of risk might be automatically determined using structured andlearned machine learning models that map the information from 430 todifferent levels of likelihood of injury.

Likelihood of Injury Distance from Risk Area None 8 Feet Low 5 FeetMedium 3 Feet High 1 Foot

Alternatively, instead of a severity or likelihood of risk, a mode forguiding the subject to an area of interest from his current location ofthe subject may be set, along with various parameters (e.g., distance tothe area of interest, direction, etc.) associated with that area.

At 450, generate control information for controlling a haptic responsebased on the severity or likelihood of injury (or activating theguidance mode) determined in operation 440. The control information maybe generated, for example, based on a correspondence or mapping definedbetween different levels of severity or likelihoods of injury andpredetermined haptic responses. This correspondence may be stored, forexample, in one or more tables in the data storage area 30 and/or in thedevice that includes the haptic actuator(s). In one case, the controlinformation may be generated for controlling activation of only oneactuator in the device worn or carried by the subject. In anotherembodiment, the control information may control activation of aplurality of actuators simultaneously or in succession, depending, forexample, on the type of alert or notification that is desired to becreated in the subject.

At 460, the control information for the haptic response is sent to thedevice worn or carried by the subject. The control information may bereceived and interpreted by a haptic feedback module, which, forexample, may be an application, processor, or other logic in the device.The haptic feedback module processes the control information todetermine (1) the type of actuator(s) to be selectively activated, (2)when to activate the actuator(s), (3) the duration of activation, (4)the intensity, frequency, or pattern of activation and/or otherinformation relating to the haptic response that is intended to becreated and perceived by the subject given the severity or likelihood ofinjury associated with his current location within the monitoring area.In some embodiments, a risk area is not involved but an area of interestto which the subject is to be guided. In this case, the actuator(s) maybe controlled, based on control information from the signal analysismodule, to generate an intended haptic response for guiding the subjectto the area of interest.

At 470, the haptic feedback module generates signals for controlling theactuators in accordance with the processed control information inoperation 460. Control of these actuators produces the hapticresponse(s) intended given the current location of the subject. In oneembodiment, the processor 10 and the haptic feedback module are in thedevice. In other embodiments, the processor is external to the deviceincluding the haptic feedback module 70. When the processor (e.g.,localization module and signal analysis module) and the haptic feedbackmodule are in the same device, the control signals may be sent to thehaptic feedback module through an internal signal path of the device.When the processor and the haptic feedback module are not in the samedevice, the control signals from the signal analysis module may be sentto the haptic feedback module over one or more wired and/or wirelesscommunication links.

In one embodiment, the system and method embodiments may be interactive.For example, the subject may verbally speak information into the deviceindicating the location within the monitoring area of where he wants togo, e.g., “I want to go to the bathroom.” A processor of the device, orthe localization module, may receive and interpret the verbal statementand then provide haptic guidance (with or without voice guidance) inorder to lead the subject to the bathroom. This may be accomplished bydetermining the current location of the subject in the monitoring areabased on the location signals, comparing the current location to areasof interest (e.g., hazards, etc.) in the layout information of themonitoring area, and then guiding the subject through the layout usinghaptic stimuli or other forms of feedback. The guidance may be performedin various ways. For example, the haptic feedback module may activateone or more actuators on a real-time basis when the subject takes awrong turn or pursues a wrong path leading away from the bathroom and/ortowards a risk area. The verbal commands received by the deviceprocessor or localization module may use, for example, natural languageprocessing in combination with a voice-first or voice-command device(e.g., Alexa, Google Home, etc.) to recognize the verbalized intent.

EXAMPLE SCENARIOS

Mobility-related risks are a key determinant of health and quality oflife. Principal among these risks is the risk of falling, which may bedivided into three broad categories of environmental risks, task-relatedrisks, and personal risks (e.g., conditions impeding mobility such asarthritis, old age, and various types of disabilities). Embodimentsdescribed herein mitigate and control environmental and task-relatedrisks and provide haptic guidance in helping those who with personalrisks.

One of the greatest risks of injury is movement along stairs or unevenor slippery surfaces. In aging populations, stairway falls pose anespecially significant risk. Individuals with vision or hearingimpairments are also at an increased risk of stairway falls. Moreover,environmental factors such as dark rooms or areas with limitedvisibility (e.g., nighttime walks to bathroom) pose additional mobilityrisks. The embodiments described herein address these problems in aunique way by providing haptic guidance that serve as warnings toindividuals of these risks before or as they are encountered.

The following descriptions are examples of scenarios in which one ormore embodiments may help a subject navigate safely in a monitoringarea. These scenarios, however, are by no means intended to be the onlyscenarios or applications of the disclosed embodiment.

Example 1

Scott is 85 years old and lives in a two-story building, where hisbedroom is located on the second floor. Scott has vision and hearingimpairments and has fallen twice in the past three months whileclimbing/descending the stairs. The risk of injury to Scott may besubstantially reduced by using a wristband device equipped with a hapticsensor module and actuators that are controlled in accordance with theembodiments described herein. For instance, when using the stairs, thehaptic feedback module may respond to signals from the signal analysismodule to deliver a single short vibration to Scott's wrist, indicatingthat he is within one stair distance to the floor. The system may alsobe configured to deliver a number of short vibrations indicating thenumber of stairs left (e.g., 3 short vibrations indicating 3 stepsleft).

Example 2

Mary is 80 years old and living with dementia, and hearing impairment.Sometimes she is unable to locate the bathroom in her home. One or moreembodiments described above can help guide Mary to the bathroom bygenerating haptic responses (in the form of directional tactile signals)using the actuators in a chest-worn pendant (or a wrist-band device) sheis wearing. In this example, the directional tactile signals may beuniquely mapped to left, right, forward and backward directions. Thismay be accomplished, for example, by positioning four actuators atdifferent locations on the pendant that correspond to respective ones ofthe directions, by pulsing or activating the actuators in differentpatterns that correspond to the different directions, or in other ways.Additionally, Mary may directly verbalize her intent to go to bathroomusing a voice-input feature. Alternatively, the system determines Mary'sintention and then directs Mary to the bathroom based on the signalsoutput from the signal analysis module mapped with information in Mary'sprofile (including toileting behavior).

Another embodiment includes a computer-readable medium storinginstructions for causing a processor to perform the operations of theembodiments described herein. For example, the instructions may causethe processor to determine a current location of a subject in amonitoring area, compare the current location to a risk area location inthe monitoring area, determine a likelihood of injury based on a resultof the comparison, and generate control information based on thelikelihood of injury to the subject. The control information may controlactivation of a haptic effect in a device worn or carried by thesubject, and the haptic effect may correspond to at least one stimulusthat notifies the subject of the potential risk area. Additionalinstructions may be stored in the computer-readable medium to performother operations of the system and method embodiments.

One or more embodiments described herein may include a number ofadditional features. For example, the system and method may beimplemented using technologies similar to those used in a PERS systemsor smartwatch devices. Also, an activity profile of the subject to bemonitored and layout information of the living environment may be usedto provide timely haptic notifications about potential hazards. In somecases, the device including the haptic feedback module and actuator(s)may be implemented on or in association with a wristband or a chest-worndevice. Different haptic feedback responses may be generated fordifferent types of risks such as climbing stairs, descending stairs,clutter, tripping hazards (e.g., loose rugs), or other objects or areasthat pose a risk to injury. Risks may also change for an individual overtime, as impairments and/or activity patterns change and new alerts maybe configured to address the new risks.

The methods, processes, and/or operations described herein may beperformed by code or instructions to be executed by a computer,processor, controller, or other signal processing device. The code orinstructions may be stored in a non-transitory computer-readable mediumin accordance with one or more embodiments. Because the algorithms thatform the basis of the methods (or operations of the computer, processor,controller, or other signal processing device) are described in detail,the code or instructions for implementing the operations of the methodembodiments may transform the computer, processor, controller, or othersignal processing device into a special-purpose processor for performingthe methods herein.

The processors, modules, units, sensors, detectors, and otherinformation generating, processing, and calculating features of theembodiments disclosed herein may be implemented in logic which, forexample, may include hardware, software, or both. When implemented atleast partially in hardware, the processors, modules, units, sensors,detectors and other information generating, processing, and calculatingfeatures may be, for example, any one of a variety of integratedcircuits including but not limited to an application-specific integratedcircuit, a field-programmable gate array, a combination of logic gates,a system-on-chip, a microprocessor, or another type of processing orcontrol circuit.

When implemented in at least partially in software, the processors,modules, units, sensors, detectors, and other information generating,processing, and calculating features may include, for example, a memoryor other storage device for storing code or instructions to be executed,for example, by a computer, processor, microprocessor, controller, orother signal processing device. Because the algorithms that form thebasis of the methods (or operations of the computer, processor,microprocessor, controller, or other signal processing device) aredescribed in detail, the code or instructions for implementing theoperations of the method embodiments may transform the computer,processor, controller, or other signal processing device into aspecial-purpose processor for performing the methods herein.

It should be apparent from the foregoing description that variousexemplary embodiments of the invention may be implemented in hardware.Furthermore, various exemplary embodiments may be implemented asinstructions stored on a non-transitory machine-readable storage medium,such as a volatile or non-volatile memory, which may be read andexecuted by at least one processor to perform the operations describedin detail herein. A non-transitory machine-readable storage medium mayinclude any mechanism for storing information in a form readable by amachine, such as a personal or laptop computer, a server, or othercomputing device. Thus, a non-transitory machine-readable storage mediummay include read-only memory (ROM), random-access memory (RAM), magneticdisk storage media, optical storage media, flash-memory devices, andsimilar storage media and excludes transitory signals.

Although the various exemplary embodiments have been described in detailwith particular reference to certain exemplary aspects thereof, itshould be understood that the invention is capable of other exampleembodiments and its details are capable of modifications in variousobvious respects. As is readily apparent to those skilled in the art,variations and modifications can be affected while remaining within thespirit and scope of the invention. Accordingly, the foregoingdisclosure, description, and figures are for illustrative purposes onlyand do not in any way limit the invention, which is defined only by theclaims.

We claim:
 1. A method for managing movement, comprising: determining acurrent location of a subject in a monitoring area; comparing thecurrent location to a risk area location in the monitoring area;determining a likelihood of injury based on a result of the comparison;and generating control information based on the likelihood of injury tothe subject, wherein the control information is to control activation ofa haptic effect in a device worn or carried by the subject and whereinthe haptic effect corresponds to at least one stimulus that notifies thesubject of the potential risk area.
 2. The method of claim 1, whereindetermining the current location includes: receiving one or morelocation signals from a corresponding number of location sensors in themonitoring area; accessing layout information corresponding to themonitoring area; comparing location information corresponding to the oneor more location signals to the layout information, and determining thecurrent location of the subject based on a result of the comparison ofthe location information to the layout information.
 3. The method ofclaim 1, further comprising: generating at least one of spatialinformation or movement information, wherein the spatial informationindicates a distance between the current location of the subject and therisk area location and wherein the movement information indicates atleast one of a direction of movement of the subject, a time of movementof the subject, or a speed of movement of the subject, and wherein thelikelihood of injury is determined based on at least one of the spatialinformation or the movement information.
 4. The method of claim 1,wherein the control information indicates at least one of: a type of ahaptic actuator in the device, a pattern of activation of the hapticactuator, a duration of activation of the haptic actuator, or a time ofactivation or deactivation of the haptic actuator.
 5. The method ofclaim 1, further comprising: accessing haptic information correlatingdifferent types of haptic effects to different likelihoods of injury,wherein the control information is generated based on the hapticinformation.
 6. The method of claim 1, wherein the risk area locationincludes a set of stairs.
 7. The method of claim 6, wherein the controlinformation controls activation of the haptic effect to alert thesubject of a remaining number of steps when the subject is on the set ofstairs.
 8. An apparatus for managing movement, comprising: a memoryconfigured to store instructions; and a processor configured to executethe instructions to: determine a current location of a subject in amonitoring area; compare the current location to a risk area location inthe monitoring area; determine a likelihood of injury based on a resultof the comparison; and generate control information based on thelikelihood of injury to the subject, wherein the control information isto control activation of a haptic effect in a device worn or carried bythe subject and wherein the haptic effect corresponds to at least onestimulus that notifies the subject of the potential risk area.
 9. Theapparatus of claim 8, wherein the processor is to determine the currentlocation by: receiving one or more location signals from a correspondingnumber of location sensors in the monitoring area; accessing layoutinformation corresponding to the monitoring area; comparing locationinformation corresponding to the one or more location signals to thelayout information, and determining the current location of the subjectbased on a result of the comparison of the location information to thelayout information.
 10. The apparatus of claim 8, wherein the processoris to: generate at least one of spatial information or movementinformation, wherein the spatial information indicates a distancebetween the current location of the subject and the risk area locationand wherein the movement information indicates at least one of adirection of movement of the subject, a time of movement of the subject,or a speed of movement of the subject, and wherein the likelihood ofinjury is determined based on at least one of the spatial information orthe movement information.
 11. The apparatus of claim 8, wherein thecontrol information indicates at least one of: a type of a hapticactuator in the device, a pattern of activation of the haptic actuator,a duration of activation of the haptic actuator, or a time of activationor deactivation of the haptic actuator.
 12. The apparatus of claim 8,wherein the processor is to access haptic information correlatingdifferent types of haptic effects to different likelihoods of injury,wherein the control information is generated based on the hapticinformation.
 13. The apparatus of claim 8, wherein the risk arealocation includes a set of stairs.
 14. The apparatus of claim 13,wherein the control information controls activation of the haptic effectto alert the subject of a remaining number of steps when the subject ison the set of stairs.
 15. The apparatus of claim 8, further comprising:one or more actuators to generate the at least one stimulus.
 16. Theapparatus of claim 15, wherein the apparatus is included in a devicecarried or worn by the subject.
 17. A computer-readable medium storinginstructions for causing a processor to: determine a current location ofa subject in a monitoring area; compare the current location to a riskarea location in the monitoring area; determine a likelihood of injurybased on a result of the comparison; and generate control informationbased on the likelihood of injury to the subject, wherein the controlinformation is to control activation of a haptic effect in a device wornor carried by the subject and wherein the haptic effect corresponds toat least one stimulus that notifies the subject of the potential riskarea.
 18. The computer-readable medium of claim 17, further storinginstructions to cause the processor is to determine the current locationby: receiving one or more location signals from a corresponding numberof location sensors in the monitoring area; accessing layout informationcorresponding to the monitoring area; comparing location informationcorresponding to the one or more location signals to the layoutinformation, and determining the current location of the subject basedon a result of the comparison of the location information to the layoutinformation.
 19. The computer-readable medium of claim 17, furtherstoring instructions to cause the processor to: generate at least one ofspatial information or movement information, wherein the spatialinformation indicates a distance between the current location of thesubject and the risk area location and wherein the movement informationindicates at least one of a direction of movement of the subject, a timeof movement of the subject, or a speed of movement of the subject, andwherein the likelihood of injury is determined based on at least one ofthe spatial information or the movement information.
 20. Thecomputer-readable medium of claim 17, wherein the control informationindicates at least one of: a type of a haptic actuator in the device, apattern of activation of the haptic actuator, a duration of activationof the haptic actuator, or a time of activation or deactivation of thehaptic actuator.